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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 

after SIX (6) MONTHS from the mailing date of this communication. . 

- If the neriod for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and wilt expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)D Responsive to communication(s) filed on . 



2a)D This action is FINAL. 2b)KI This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) S Claim(s) 1-22 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) M Claim(s) 1^22 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)D All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) Q Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 

1) Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) M Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-152) 

3) ^ Information Disclosure Statement(s) (PTO-1449) Paper No(s) 2 6) □ Other: 
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PTO-326 (Rev. 04-01) Office Action Summary Part of Paper No. 3 
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DETAILED ACTION 



Claim Rejections - 35 USC § 103 



1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 

rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1-22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Eppler, U.S. 
Pat. No. 6,084,989 in view of Szeliski et al., U.S. Pat. No. 6,157,747 

Considering claim 1, Eppler discloses the following claimed subject matter, note; 

a) the claimed method of extracting a sequence of video frames is met by Patch Extractor 17, 
FIG.2, which receives image frame and extracts an image patch. 

b) the claimed method of sampling each of the video frames is met Image Restoration 24 5 FIG.2; 
C) the claimed method of interpolating the up sampled video frames is met by Statistics 
computation and gray scale enhancement 25, FIG. 2, which "forms an enhanced version of the 
image E(c,r) which is input to selected ones of the matching algorithms 23 for processing to 
compute the offset errors." (Col. 6, lines 34-36) 



Except for; 



d) the claimed method of aligning the interpolated video frames; 
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e) the claimed method of creating a single image from the aligned video frames; 

Regarding d), Eppler doesn't specifically use the term alignment. However, Eppler 
discloses that "...The up. sampled boundary and area data are processed by one or more matching 
algorithms 23 that produce column and row offset errors and a figure of merit regarding the 
match between the predicted landmark and the actual landmark in the image." (Col. 6, lines 15- 
20) Eppler also discloses that "Six different image matching algorithms 23 were investigated 
...The image matching algorithms 23 include centroid image matching, edge matching, cluster 
reward, minimum variance, cross correlation, and normalized cross correlation algorithms 23." 
(Col. 6, lines 43-49) t^U* 1° » I 7 

Szeliski et al., discloses a 3 -dimensional image rotation method and apparatus for 
producing image mosaics which invention "aligns a set of plural images to construct a mosaic 
image", (see Abstract) Specifically, Szeliski et al, discloses image alignment using patch-based 
algorithm, 120 (FIG.l). 

Therefore, it would have been obvious to the skilled in the art at the time the invention 
was made to readily recognize the matching algorithms 23 of Eppler and the image alignment 
(using patch-based algorithm) of Szeliski et al. and replace/provide the latter in the system of 
Eppler and modify the same. 

Regarding e), see rejection of (d). 
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Considering claim 2, the claimed method wherein the sequence of video frames are low resolution 
images is inherent, because the system produces a "higher resolution display generated" (col. 13, 
lines 16-22 after the images have been processed. 

Considering claim 3, the claimed method wherein said up sample step is by a factor of 4 is met by 
image restoration (up sample 7x4), FIG.2; (see also col. 14, lines 52-56) 

Considering claim 4, the claimed method wherein said align step comprises aligning the video 
images in an x direction and a y direction in a center portion of interest in each video frame is 
inherent, because alignment of images entails in all directions. 

Considering claim 5, the claimed comprising extracting the sequence of video frames at 30 
frames/sec is inherent, because the rate of 30 frames/sec is standard in video/television systems. 

Considering claim 6, the claimed wherein the sequence of video frames includes 5 video frames. 

Regarding claim 6, Eppler doesn't specifically disclose a sequence of video frames 
including 5 video frames. However, it would be obvious to the skilled in the art that this is a 
design choice. 
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Considering claim 7, the claimed method of correlating the up sampled video images is met by the 
"cross correlation algorithm 23" (col. 6, 48-49) 

Considering claim 8, the claimed rising averaging a pixel intensity from each of the up sampled 
video frames is met by the disclosure that "The average and the variance are computed for a all 
gray-scale pixel values with the landmark perimeter." (Col. 9, lines 31-33) 

Considering claim 9, the claimed comprising compensating for platform movement and rotation 
zoom is met by the disclosure that "...Eliminating scale factor and rotation errors preserves the 
sharpness of the correlation surface peak and improves the reliability and accuracy of image 
matching." (Col. 15, lines 24-27) 

Considering claim 10, the claimed aligning each the extracted sequence of video frames before 
said up sample step; 

Regarding claim 10, Eppler up samples and performs matching algorithm. However, it 
would be a design choice whether to up sample (magnify) the images after or before alignment. 
Therefore, it would have been obvious to the skilled in the art at the time the invention was made 
to readily recognize the teaching of the prior art and provide an alignment after or before the up 
sample step. 
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Considering claim 11, The method of claim 1, comprising identifying commonality from one 
individual frame to the next and overlapping the individual frames and displaying an image 
representing a continuous area. 

Regarding claim 11, Eppler doesn't specifically disclose overlapping the individual frames. 
However, Eppler discloses the image patch that uses Matching algorithms 23. Szeliski et al. 
discloses image alignment utilizing patch-based algorithm. Therefore, it would have been 
obvious to the skilled in the art at the time the invention was made to readily recognize the 
teaching of Eppler and provide the system of Szeliski to modify the system of Eppler in order to 
provide the capacity of overlapping the individual frames. 

Considering claim 12, The method of claim 11, comprising extracting the sequence of video 
frames at 30 frames/sec is inherent, because the rate of 30 frames/sec is standard in television 
systems. 

Considering claim 13, the claimed wherein the sequence of video frames includes 5 video frames. 

Regarding claim 6, Eppler doesn't specifically disclose a sequence of video frames to 
include 5 video frames. However, it would be obvious to the skilled in the art that this would be a 
design choice to specify a number of video frames in the a sequence 
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Considering claim 14, the claimed correlating the up sampled video images is met by the "cross 
correlation algorithm 23" (col. 6, 48-49) 

Considering claim 15, the claimed averaging a pixel intensity from each of the up sampled video 
frames. 

Regarding claim 15, see rejection of claim 8. 

Considering claim 16, The claimed comprising compensating for platform movement and rotation 
zoom. 

Regarding claim 16, see rejection of claim 9. 

Considering claim 17, the claimed method of claim 16, comprising aligning each the extracted 
sequence of video frames before said up sample step. 
Regarding claim 17, see rejection of claim 10. 

Considering claim 18, The method of claim 17, comprising identifying commonality from one 
individual frame to the next and overlapping the individual frames and displaying an image 
representing a continuous area. 

Regarding claim 1 8, see rejection of claim 1 1 . 
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Considering claim 19, see rejection of claim 5. 
Considering claim 20, see rejection of claim 1 . 
Considering claim 21, see rejection of claim 1. 

Considering claim 22, Eppler discloses the following claimed subject matter, note; 

a) the claimed processor is met by the computer-system 16, FIG. 1 ; 

b) the claimed memory coupled to said processor is inherent in computer-systems. 

c) the claimed memory having stored therein sequences of instructions when executed by said 
processor, causes said processor to perform the steps is met by the disclosure in FIG.4. (See also 
rejection of claim 1). 

d) the claimed steps of extract a sequence of video frames; up sample each of the video frames; 
interpolate the up sampled video frames; align the interpolated video frames; and create a single 
image from the aligned video frames; 

Regarding d), see rejection of claim 1 . 
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Conclusion 



The prior art made of record and not relied upon is considered pertinent to applicant's 



disclosure. 

Burt et al. , U.S. Pat. No. 5,649,032 discloses a system for automatically aligning images to form 
a mosaic image. 

Any inquiry concerning this communication or earlier 
communications from the examiner should be directed to Paulos 
Natnael whose telephone number is (703) 305-0019. The examiner 
can normally be reached on Monday through Thursday from 8:00 a. 
M. to 5:00 p.m. The examiner can also be reached on alternate 
Fridays . 

If attempts to reach the examiner by telephone are 
unsuccessf ul, the examiner's supervisor, John Miller, can be 
reached on (703)305-4795. 

Any inquiry of a general nature or relating to the status of 
this application or proceeding should be directed to the Group 
receptionist whose telephone number is (703)305-3900. 
Any response to this action should be mailed to: 



Commissioner of Patents and Trademarks 



Washington, D.C. 20231 



# 



• 
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or faxed to: 

(703) 872-9314, (for formal communications intended for 
entry) 

or; 

(703)872-9314 (for informal or draft communications, 
please label "PROPOSED" OR "DRAFT"). 
Hand-delivered responses should be brought to Crystal Park 
II, 2121 Crystal Drive, Arlington, V.A. Sixth Floor 
(Receptionist) . 

Paulos Af. Natnael 

March 7, 2002 





JOHN W. MILLER 
rHWENT EXAMINER 



